EOSC-SYNERGY: Una iniciativa para la implementación del EOSC liderada por España

The presentation shows the last trends in services of cloud computing for research data. It details the European Open Science Cloud and its Spanish implementatio

EOSC-SYNERGY: Una iniciativa para la implementación del EOSC liderada por España

The presentation shows the last trends in services of cloud computing for research data. It details the European Open Science Cloud and its Spanish implementatio

An integrated IaaS and PaaS architecture for scientific computing

Trabajo presentado al EGI Community Forum, celebrado en Bari (Italia) del 10 al 13 de noviembre de 2015.Scientific applications often require multiple computing resources deployed on a coordinated way. The deployment of multiple resources require installing and configuring special software applications which should be updated when changes in the virtual infrastructure take place. When working on hybrid and federated cloud environments, restrictions on the hypervisor or cloud management platform must be minimised to facilitate geographic-wide brokering and cross-site deployments. Moreover, preserving the individual operation at the site-level in federated clouds is also important for scalability and interoperability. In that sense, the INDIGO-DataCloud project [1] has been designed with the objective of building up a PaaS-level cloud solution for research. One of the key multi-level components is the PaaS computing core. This part constitutes the kernel for the deployment of services and computing virtual infrastructures for the users. It is complemented with the virtualized storage, federated AAI and networking. The INDIGO-DataCloud PaaS core will be based on a microservice architecture [2]. Microservices consist of a set of narrowly focused, independently deployable services, typically implemented using container-embedded applications, exposed by RESTful interface. Microservices are designed to be highly scalable, highly available and targeted for the use in cloud environments. INDIGO's microservices will be deployed, dynamically scheduled and managed using tools such as kubernetes [3]. In cases where multi-tenancy is not yet intrinsically supported by the particular microservice, like the container manager, INDIGO-DataCloud may decide to offer multiple instances to bridge that gap. INDIGO PaaS will offer an upper layer orchestration service for distributed applications using the TOSCA language standard [4]. It will deal with the requested service instantiation and application execution, managing the needed microservices in order, for example, to select the right end-point for the deployment. Cross-site deployments will also be possible. This PaaS, aimed at providing a more efficient platform for scientific computing, will require additional characteristics from the underlying layers. The INDIGO PaaS will leverage an enhanced IaaS that will provide a richer set of features currently missing. The usage of TOSCA permits IaaS providers to offer infrastructure orchestration, making possible to manage the deployment and configuration of the resources that are being provided. The life-cycle of the resources is therefore managed through the APIs exposed by the IaaS end-points. The TOSCA templates will be translated into their native deployment schemas using IM [5] for OpenNebula and Heat-Translator [6] for OpenStack HEAT. Both OpenNebula and OpenStack will incorporate drivers to support the deployment of containers as first-class resources on the IaaS. This will provide high efficiency when building up complex configurations from a repository of container images. The scheduling algorithms for both cloud management frameworks will be improved, in order to provide a better experience for the end-users and a more efficient utilization of the computational resources. The usage of two-level orchestrator (at the level of PaaS and within each IaaS instances) will enhance the capabilities of providing a dynamic and on-demand increase in cloud resourcesPeer Reviewe

Red española de e-Ciencia: impulsando la actividad científica española mediante el uso colaborativo de recursos distribuidos

Reunión de RECOLECTAUCM. BibliotecaFALSEunpu

A Parallel Implementation of the K Nearest Neighbours Classifier in Three Levels: Threads MPI Processes and the Grid

The work described in this paper tackles the problem of data mining and classification of large amounts of data using the K nearest neighbours classifier (KNN) [1]. The large computing demand of this process is solved with a parallel computing implementation specially designed to work in Grid environments of multiprocessor computer farms. The different parallel computing approaches (intra-node, inter-node and inter-organisations) are not sufficient by themselves to face the computing demand of such a big problem. Instead of using parallel techniques separately, we propose to combine the three of them considering the parallelism grain of the different parts of the problem. The main purpose is to complete a 1 month-CPU job in a few hours. The technologies that are being used are the EGEE Grid Computing Infrastructure running the Large Hadron Collider Computing Grid (LCG 2.6) middleware [3], MPI [4] [5] and POSIX [6] threads. Finally, we compare the results obtained with the most popular and used tools to understand the importance of this strategy.Aparicio Pla, G.; Blanquer Espert, I.; Hernández García, V. (2007). A Parallel Implementation of the K Nearest Neighbours Classifier in Three Levels: Threads MPI Processes and the Grid. En High Performance Computing for Computational Science - VECPAR 2006. Springer Verlag (Germany). 225-235. doi:10.1007/978-3-540-71351-7_18S225235Cover, T.M., Hart, P.E.: Nearest neighbour pattern recognition. IEEE Trans. on Information Theory 13(1), 2127 (1967)Foster, I., Kesselman, C., Tuecke, S.: The Anatomy of the Grid: Enabling Scalable Virtual Organizations. International J. Supercomputer Applications 15(3) (2001), http://www.globus.org/research/papers/anatomy.pdfLCG: World Wide Web Computing Grid. Distributed Production Environment of Physics Data Processing. http://lcg.web.cern.ch/LCGMessage Passing Interface Forum: MPI: A message-passing interface standard (2003), http://www.mpi-forum.org/Gropp, W., et al.: MPI: The Complete Reference. MIT Press, Cambridge (1998)Drepper, U., Molnar, I.: The Native POSIX Thread Library for Linux (2003), http://people.redhat.com/drepper/nptl-design.pdfFrank, E., Hall, M., L.T.: Weka 3: Data Mining Software in Java (2005), http://www.cs.waikato.ac.nz/ml/wek

Vertical Elasticity on Marathon and Chronos Mesos frameworks

[EN] Marathon and Chronos are two popular Mesos frameworks that are widely used for deploying fault-tolerant services and periodic batch jobs. Marathon and Chronos provide by design mechanisms for horizontal elasticity, scaling up and down the number of job and service instances. Horizontal elasticity is appropriate when the problems that are solved are inherently parallel. However, when the problem cannot benefit from an increase of the amount of resources, vertical elasticity must be considered. This work implements on top of Marathon and Chronos Mesos frameworks, a mechanism to vary the resources associated to an executor dynamically, according to its progress and considering specific Quality of Service (QoS). The mechanism developed provides a wrapper executable and a service that takes the decision of increasing or decreasing the resources allocated to different Chronos iterations or a long-living Marathon application. The mechanism makes use of checkpointing techniques to preserve the execution of Marathon applications and leverages OpenStack Monasca for the monitoring. \footnote{The work in this article has been funded by projects BIGCLOE and EUBra-BIGSEA, BIGLOE is funded by the Spanish ``Ministerio de Econom\'ia, Industria y Competitividad" with reference number TIN2016-79951-R and EUBra-BIGSEA is funded jointly by the European Commission under the Cooperation Programme, Horizon 2020 grant agreement No 690116 and the Brazilian Ministério de Ciência, Tecnologia e Inovação (MCTI).The work in this article has been funded by projects BIGCLOE and EUBra BIGSEA, BIGLOE is funded by the Spanish "Ministerio de Economia, Industria y Competitividad" with reference number TIN2016-79951-R and EUBra-BIGSEA is funded jointly by the European Commission under the Cooperation Programme, Horizon 2020 grant agreement No 690116 and the Brazilian Ministerio de Ciencia, Tecnologia e Inovacao (MCTI).López-Huguet, S.; Natanael, I.; Brito, A.; Blanquer Espert, I. (2019). Vertical Elasticity on Marathon and Chronos Mesos frameworks. Journal of Parallel and Distributed Computing. 133:179-192. https://doi.org/10.1016/j.jpdc.2019.01.002S17919213

SLA-Driven Cloud Computing Domain Representation and Management

The assurance of Quality of Service (QoS) to the applications, although identified as a key feature since long ago [1], is one of the fundamental challenges that remain unsolved. In the Cloud Computing context, Quality of Service is defined as the measure of the compliance of certain user requirement in the delivery of a cloud resource, such as CPU or memory load for a virtual machine, or more abstract and higher level concepts such as response time or availability. Several research groups, both from academia and industry, have started working on describing the QoS levels that define the conditions under which the service need to be delivered, as well as on developing the necessary means to effectively manage and evaluate the state of these conditions. [2] propose Service Level Agreements (SLAs) as the vehicle for the definition of QoS guarantees, and the provision and management of resources. A Service Level Agreement (SLA) is a formal contract between providers and consumers, which defines the quality of service, the obligations and the guarantees in the delivery of a specific good. In the context of Cloud computing, SLAs are considered to be machine readable documents, which are automatically managed by the provider's platform. SLAs need to be dynamically adapted to the variable conditions of resources and applications. In a multilayer architecture, different parts of an SLA may refer to different resources. SLAs may therefore express complex relationship between entities in a changing environment, and be applied to resource selection to implement intelligent scheduling algorithms. Therefore SLAs are widely regarded as a key feature for the future development of Cloud platforms. However, the application of SLAs for Grid and Cloud systems has many open research lines. One of these challenges, the modeling of the landscape, lies at the core of the objectives of the Ph. D. Thesis.García García, A. (2014). SLA-Driven Cloud Computing Domain Representation and Management [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/36579TESI

An OGSA Middleware for Managing Medical Images Using Ontologies

The final publication is available at Springer via http://dx.doi.org/ 10.1007/s10877-005-0675-0This article presents a Middleware based on Grid Technologies that addresses the problem of sharing, transferring and processing DICOM medical images in a distributed environment using an ontological schema to create virtual communities and to define common targets. It defines a distributed storage that builds-up virtual repositories integrating different individual image repositories providing global searching, progressive transmission, automatic encryption and pseudo-anonimisation and a link to remote processing services. Users from a Virtual Organisation can share the cases that are relevant for their communities or research areas, epidemiological studies or even deeper analysis of complex individual cases. Software architecture has been defined for solving the problems that has been exposed before. Briefly, the architecture comprises five layers (from the more physical layer to the more logical layer) based in Grid Thecnologies. The lowest level layers (Core Middleware Layer and Server Services layer) are composed of Grid Services that implement the global managing of resources. The Middleware Components Layer provides a transparent view of the Grid environment and it has been the main objective of this work. Finally, the upest layer (the Application Layer) comprises the applications, and a simple application has been implemented for testing the components developed in the Components Middleware Layer. Other side-results of this work are the services developed in the Middleware Components Layer for managing DICOM images, creating virtual DICOM storages, progressive transmission, automatic encryption and pseudo-anonimisation depending on the ontologies. Other results, such as the Grid Services developed in the lowest layers, are also described in this article. Finally a brief performance analysis and several snapshots from the applications developed are shown. The performance analysis proves that the components developed in this work provide image processing applications with new possibilities for large-scale sharing, management and processing of DICOM images. The results show that the components fulfil the objectives proposed. The extensibility of the system is achieved by the use of open methods and protocols, so new components can be easily added.Blanquer Espert, I.; Hernández García, V.; Segrelles Quilis, JD. (2005). An OGSA Middleware for Managing Medical Images Using Ontologies. Journal of Clinical Monitoring and Computing. 19:295-305. doi:10.1007/s10877-005-0675-0S29530519“European DataGrid Project”. http://www.eu-datagrid.org.“Biomedical Informatics Research”. http://www.nbirn.net/.“ACI project MEDIGRID: medical data storage and processing on the GRID”.http://www.creatis.insa-lyon.fr/MEDIGRID/.“Information eXtraction from Images (IXI) Grid Services for Medical Imaging”. Working Notes of the Workshop on Distributed Databases and processing in Medical Image Computing (DIDAMIC'04). Pag 65.“NeuroBase: Management of Distributed and Heterogeneous Information Sources in Neuroimaging”. Working Notes of the Workshop on Distributed Databases and processing in Medical Image Computing (DIDAMIC'04). Pag 85.Digital Imaging and Communications in Medicine (DICOM) Part 10: Media Storage and File Format for Media Interchange. National Electrical Manufacturers Association, 1300 N. 17th Street, Rosslyn, Virginia 22209 USA.“Open Grid Services Architecture (OGSA)”, http://www.globus.org/ogsa.Globus alliance Home Page. “Relevant documents”, http://www.globus.orgAllen Wyke R, Watt A, “XML Schema Essentials”. Wiley Computer Pub. ISBN 0-471-412597Web security and commerce/Simson Garfinkel. - Cambridge: O'Reilly, 1997. - 483 p.; 23 cm. ISBN 1565922697“The GridFTP Protocol and Software”. http://www-fp.globus.org/datagrid/gridftp.html.JPEG2000: Image compression fundamentals, standards and practice/David S. Taubman, Michael W. Marcellin. – Boston [etc.] : Kluwer Academic, cop. 2002. - XIX, 773 p.; 24 cm. + 1 CD-Rom - (The Kluwer international series in engineering and computer science) ISBN 079237519XBradley J, Erickson MD, “Irreversible Compression of Medical Images”, Dpt. Radiology, Mayo F., Rochester, MN, Jo. of D. Imaging, DOI: 10.1007/s10278-002-0001-z, 02.Monitoring & Discovery System (MDS)” http://www-unix.globus.org/toolkit/mds/“Key management for encrypted data storage in distributed systems”. Proceedings of HeathGrid 2004

Analysis on the efects of turbulent inflow conditions on spray primary atomization in the near-field by direct numerical simulation

[EN] It is widely acknowledged that the development of sprays in the near-field is of primary importance for the spray formation downstream, as it affects both the spray angle, as well as the intact core length. In this frame, the present work aims to study the effects of turbulence inlet boundary condition on the spray formation by means of Direct Numerical Simulations on a real condition at low Reynolds number. To this extent, the code Paris-Simulator has been used, while a digital filter-based algorithm was used in order to generate synthetic turbulence at the inlet boundary condition. The influence of turbulence intensity and lengthscale on the atomization process has been studied and analyzed through 3 simulation for which these parameters have been varied. The results clearly highlight how the atomization is heavily affected by the inlet turbulence configuration. An analysis of the different atomizing conditions has been conducted, aiming to understand how the variation introduced by the inlet boundary condition on the velocity field is affecting the local atomization dynamics.This work was partly sponsored by "Ministerio de Economia y Competitividad", of the Spanish Government, in the frame of the Project "Estudio de la interaccion chorro-pared en condiciones realistas de motor", Reference TRA2015-67679-c2-1-R. The author thankfully acknowledges the computer resources at MareNostrum (BSC) and the technical support provided by FI-2016-3-0031.Salvador, FJ.; Ruiz, S.; Crialesi Esposito, M.; Blanquer Espert, I. (2018). Analysis on the efects of turbulent inflow conditions on spray primary atomization in the near-field by direct numerical simulation. International Journal of Multiphase Flow. 102:49-63. https://doi.org/10.1016/j.ijmultiphaseflow.2018.01.019S496310